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Introduction 

At  the  present  time  the  Naval  Air  Warfare  Center  (NAWC),  Aircraft  Division,  Warminister, 
Software  &  Computer  Technology  Division  (SCTD),  is  developing  software  to  solve  the  problem 
of  multiple  objective  task  allocation  to  multicomputer  nodes  optimally.  This  type  of  optimization 
problem  is  classified  as  an  NP-complete  problem  of  exponential  time  complexity.  Two  algorithms 
are  being  used  in  the  software:  the  genetic  algorithm  and  the  simulated  annealing  algorithm.  The 
eventual  goal  of  this  project  is  to  allow  the  decision  maker  (DM)  to  have  options  available  to  her/ 
him  when  designing  networks  of  allocating  linked  software  tasks  to  linked  computer  hardware 
nodes. 

In  terms  of  modeling  the  problem,  the  approach  is  to  create  one  objective  function  from  a  weighted 
sum  of  all  the  individual  objective  functions.  SCTD  is  using  task  precedence  graphs  to  model  the 
problem  rather  than  using  a  task  interaction  graph  mapping.  This  problem  can  be  classified  as  a 
combinatorial  optimization  problem  concerned  with  finding  the  best  or  optimal  solution  among  a 
finite  or  countably  infinite  number  of  alternative  solutions  (Sheddin). 

The  language  implementation  is  in  Ada,  using  a  linked  list  structure. 

Literature  Search 

A  search  of  the  literature  reveals  that  abundant  research  is  in  progress  with  respect  to  the  use  of  the 
genetic  algorithms  and  simulated  annealing  algorithms  in  the  solving  of  optimization  problems. 
Another  technique  is  also  gaining  much  attention  and  that  technique  is  the  use  of  artificial  neural 
networks  to  solve  optimization  problems. 

Fox  and  Furmanski  (1988)  developed  a  “bold  neural  network”  algorithm  for  balancing  the  load  for 
mapping  loosely  synchronous  problems  onto  hypercubes.  Neural  networks  had  been  used  by 
Hopfield  and  Tank  (1986)  for  the  solution  of  optimization  problems  such  as  the  traveling  salesman 
(TSP).  Their  work  showed  that  neural  networks  are  fast,  reliable,  and  easy  to  implement. 

Mansour  and  Fox  (1991)  have  done  experiments  involving  the  use  of  genetic  (GA),  simulated 
annealing  (SA),  and  neural  network  (NN)  algorithms  applied  to  different  problem  sets.  The 
problem  sets  involve  allocating  data  to  multicomputer  nodes.  These  experiments  have  compared 
the  three  algorithms  with  respect  to  solution  quality,  execution  time,  bias,  parallelizability,  and 
robustness.  Their  results  indicated  that  the  genetic  and  simulated  annealing  algorithms  give  good 
solutions  that  are  close  to  optimal  but  the  algorithms  are  very  slow  for  problems  of  moderate  size. 
However,  the  neural  network  algorithm  does  not  give  as  good  a  solution  as  the  genetic  or  simulated 
annealing  algorithms  but  the  algorithm  is  much  faster  than  the  other  two  algorithms.  The  neural 
network  algorithm  also  tended  to  favor  certain  types  of  topologies.  They  use  an  approximate 
objective  function  that  is  smooth  and  consists  of  a  weighted  sum  of  individual  objectives.  This 
approximate  objective  function  overcomes  the  non-smooth  objective  function  that  applies  to  the 
problem.  The  two  goals  of  the  optimization  are  load  balancing  and  minimum  communication. 
Their  implementation  language  is  FORTRAN. 

Mansour  and  Fox  (1992)  evolve  these  three  algorithms  (GA,  SA,  and  NN)  into  parallel  versions. 
The  parallel  versions  of  these  algorithms  are  applied  to  allocating  irregular  or  general  data  to 
multicomputer  nodes  such  that  total  execution  time  is  minimized.  The  data  allocation  problem  is  an 
NP-complete  resource  allocation  problem.  The  parallization  of  the  algorithms  is  needed  for  large  or 
dynamically  varying  problems.  This  effort  was  part  of  a  larger  FORTRAN  D  project. 
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The  three  parallel  algorithms  were  compared  for  solution  quality,  bias,  execution  time,  scalability, 
robustness,  and  memory  space  requirements.  Comparisons  of  the  three  parallel  algorithms  consistly 
showed  the  genetic  version  producing  the  best  solutions,  with  simulated  annealing  second  best,  and 
neural  network  solutions  last.  With  respect  to  execution  time  the  genetic  version  is  the  slowest  and 
the  neural  network  version  is  the  fastest. 

The  parallel  algorithms  were  less  robust  than  their  sequential  counterparts. 

For  large  problems  the  genetic  version  required  large  memory  space.  The  authors  indicate  that  this 
problem  of  large  memory  space  requirement  could  be  overcome  by  adding  a  preprocessing  graph 
contraction  step  to  thealgorithm.  This  might  lead  to  a  decrease  in  solution  quality,  but  it  would  lead  to 
a  significant  decrease  in  execution  time. 

The  parallel  versions  of  simulated  annealing  and  neural  networks  are  scalable.  The  genetic  version 
could  be  scalable,  also,  if  the  above  suggestion  of  circumventing  memory  space  restriction  was 
implemented. 

At  NAWC,  the  Air  Vehicles  and  Crew  Systems  Technology  Department,  Steinberg  and  DiGirolamo 
(1 991 )  are  using  neural  networks  to  develop  flight  control  systems  with  a  goal  to  prove  that  neural 
network  based  approaches  are  feasible  in  meeting  flight  safety  requirements  for  implementation  on 
manned  aircraft.  They  are  using  neural  networks  to  resolve  the  complex  design  of  flight  control 
systems  (FCS)  and  to  create  designs  with  better  performance  for  less  labor  and  cost  Their 
implementation  languages  are  FORTRAN  and  C++.  They  do  not  use  Ada  in  their  work. 

Freeman  and  Skapura  (1991),  working  with  neural  networks,  recommend  the  use  of  network  data 
arranged  in  groups  of  linearly  sequential  arrays,  since  it  is  much  faster  to  step  through  an  array 
sequentially  than  it  is  to  look  up  the  address  of  every  new  value  as  would  be  done  if  a  lmked-list 
approach  were  used.  They  give  several  examples  using  pseudo-code  and  data  structure  diagrams 

Multi-objective  Functions 

The  discussion  so  far  has  concentrated  on  the  algorithms  to  use  to  solve  task  allocation  problems.  We 
now  turn  to  the  problem  of  more  than  one  objective  to  optimize,  or  the  optimization  of  multiple 
objectives  Wann  et  al  ( 1 992)  have  taken  the  approach  of  optimizing  a  weighted  sum  of  the  individual 
objective  functions.  Warburton  (1987),  working  with  methods  for  approximating  the  set  of  Pareto 
optima  paths  in  multiple  objective,  shortest-path  problems,  cautions  that  using  a  weighted  linear 
combination  of  objective  functions  to  approximate  the  optimal  may  miss  large  sections  of  efficient 
sets  of  acceptable  solutions  and  we  do  not  know  which  sections  are  unknown.  Pareto  optimality  is  the 
set  of  efficient  answers  that  are  not  dominated  by  others.  Cohen  ( 1 992)  (referring  to  Warburton)  and 
Warburton  (1987)suggest  using  a  vector  of  objective  functions  approach  to  the  optimization 
problem.  The  approximation  methods  are  fully  polynomial. 

Paul  ( 1 990)  worked  on  the  problem  of  assigning  naval  officers  to  certain  billets  using  a  hierarchy  of 
multiple  objectives.  His  problem  was  the  classical  0-1  assignment  problem.  His  technique  for  solving 
this  problem  was  to  create  a  hierarchy  of  the  objectives  and  solve  various  subproblems  using  the 
objectives  eventually  as  constramts  in  the  subproblems.  The  first  subproblem  is  solved  using  the 
highest  priority  objective  function  only.  The  second  subproblem  is  solved  using  the  second  highest 
priority  objective  function,  but  with  the  first  highest  objective  function  now'  used  as  a  constraint  (or  a 
high  percentage  of  it).  Continue  in  this  manner  for  each  additional  objective  function  with  each  higher 
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objective  function  used  as  one  of  the  constraints.  The  process  is  finished  when  all  objective 
functions  have  been  exhausted.  He  used  the  simulated  annealing  algorithm  and  obtained  reason¬ 
able  results  on  problems  with  small  search  spaces. 

Ercal  (1988)  addressed  the  problem  of  static  task  mapping  and  used  task  interaction  graphs  to 
model  the  mapping  of  iterative  parallel  programs  onto  processors.  His  problem  involved  two  goals: 
1)  even  distribution  of  total  computational  load  among  the  processors,  and  2)  minimization  of 
interprocessor  communication.  He  used  two  different  approaches  for  comparison:  graph-based 
heuristic  approaches  (nearest-neighbor  approach  and  mincut-based  clustering  schemes)  and  modi¬ 
fied  simulated  annealing.  Comparisons  were  also  made  with  a  greedy  algorithm.  The  results  of  his 
work  indicated  that  the  modified  simulated  annealing  algorithm,  with  regard  to  solution  quality  and 
running  time  fell  in  between  graph-based  heuristic  approach  and  the  greedy  method. 

Bollinger  and  Midkiff  (1988)  worked  on  the  problem  of  mapping  a  set  of  processes  and  their 
communication  requirements  onto  a  multicomputer.  They  did  not  consider  the  problem  in  which 
the  number  of  processes  exceeds  the  number  of  processors.  The  multiple  objectives  of  load  balance 
and  minimizing  interprocessor  communication  traffic  over  network  links  were  combined  into  one 
objective  using  a  weighted  sum  of  the  individual  objectives.  A  weight  was  attached  to  one 
objective  such  that  that  objective  would  be  heavily  penalized  if  it  increased  during  the  optimality 
process. 

Zoints  and  Wallenius  (1976)  used  an  interactive  mathematical  programming  method  for  solving 
the  multiple  criteria  problem  involving  a  single  decision  maker.  The  basis  of  their  method  is  the 
feasibility  of  using  linear  approximations  to  represent  the  set  of  constraints  and  objective  functions. 
Their  method  was  to  create  a  composite  objective  function  (or  utility  function)  using  multipliers 
(weights).  The  composite  objective  function  was  optimized  to  produce  an  efficient  or  Pareto- 
optimal  solution  to  the  problem.  An  example  (profits  of  a  firm),  in  which  the  utility  function  is  not 
linear,  is  given  with  the  objective  modified  to  retain  the  assumption  of  linearity. 

In  their  model  the  sum  of  the  weights  is  one.  The  weights  are  chosen  such  that  it  is  not  possible  to 
increase  (decrease)  one  objective  without  decreasing  (increasing)  at  least  one  other  objective 
function. 

Loganathan  and  Sherali  (1987)  discussed  different  classifications  of  the  multiobjective  optimiza¬ 
tion  problem.  The  first  type  has  the  task  of  characterizing  the  set  of  efficient,  or  nondominated,  or 
Pareto  optimal,  solutions.  These  solutions  are  such  that  no  other  solution  can  improve  even  one 
objective  function  without  worsening  the  others.  The  task  of  selecting  among  such  solutions  is 
typically  left  unaddressed.  Here  the  knowledge  of  the  decision  maker  is  not  used. 

At  the  other  extreme  the  decision  maker’s  preferences  are  specified  as  in  goal  programming  or 
multiattribute  utility  theory.  Since  the  decision  maker’s  utility  function  is  explicitly  known,  the 
problem  is  not  a  true  multiobjective  program,  since  it  may  be  solved  as  a  single  scalar  problem. 

A  third  approach,  and  intermediate  type  of  approach,  is  one  that  uses  partial  progressively  revealed 
information  regarding  the  decision  maker’s  preferences.  This  approach  is  known  as  an  interactive 
method  because  it  seeks  information  progressively  from  the  decision  maker.  These  methods 
typically  seek  to  solve  a  sequence  of  subproblems. 
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Definitions 

Goldberg  (1989)  defines  the  genetic  algorithm  as  a  search  algorithm  based  on  the  mechanics  of 
natural  selection  and  natural  genetics. 

Freeman  and  Skapura  (1991)  define  neural  networks  (also  known  as  connection  networks)  as  a 
collection  of  parallel  processors  connected  together  in  the  form  of  a  directed  graph,  organized  such 
that  the  network  structure  lends  itself  to  the  problem  being  considered. 

Shedden:(1990)  defines  simulated  annealing  as  a  general  purpose  approximation  algorithm 
applicable  to  many  combinatorial  optimization  problems.  The  algorithm  behaves  like  a  local  serach 
that  probabilistically  accepts  transitions  away  from  a  optimality,  thus  allowing  it  to  escape  local 
optima  in  its  search  for  the  global  optimum.  This  algorithm  is  independent  of  initial  configurations 

The  combinatorial  optimation  problem  is  concerned  with  finding  the  best  or  optimal  solution  among  a 
finite  or  countably  infinite  number  of  alternative  solutions. 

Chen  (1990)  states  that  simulated  annealing  is  derived  from  Monte  Carlo  methods  in  statistical 
mechanics  and  is  a  stochastic  optimization  algorithm  that  simulates  the  annealing  process  The 
algorithm  was  first  utilized  as  a  simulation  method  to  examine  the  properties  of  substances  consisting 
of  interacting  individual  molecules.  The  purpose  of  the  simulation  is  to  find  the  ground  states  of  a 
system  which  corresponds  to  the  configurations  of  low  energy  molecular  structure. 

Simulated  annealing  has  a  goal  of  finding  the  minimum  of  a  function  of  many  variables  by  first  heating 
the  system  being  optimized  at  a  high  temperature  and  then  lowering  the  temperature  in  slow  stages 
until  the  system freezes  The  sequence  of  temperatures,  including  the  melting  point  and  freezing  point, 
is  called  the  annealing  schedule. 

Blower  ( 1 990)  says  that  simulated  annealing  has  a  goal  of  finding  the  minimum  of  a  function  of  many 
variables  by  first  heating  the  system  being  optimized  at  a  high  temperature  and  then  lowering  the 
temperature  in  slow  stages  until  the  system  freezes  The  sequence  of  temperatures,  including  the 
melting  point  and  freezing  point,  is  called  the  annealing  schedule. 

Simulated  annealing  is  a  stochastic-search  technique  with  a  control  parameter  represented  by  the 
temperature.  The  algorithm  attempts  to  evaluate  a  complicated  multidimensional  integral  by  Monte 
Carlo  sampling  approach.  The  Monte  Carlo  technique  is  necessary  because  the  integral  is  too 
complicated  to  solve  through  conventional  analytical  means.  Because  of  the  complicated  nature  of 
the  integral,  the  Monte  Carlo  approach  follows  a  scheme  designed  to  provide  better  sampling  of  the 
states  where  major  contributions  to  the  integral  are  made. 

Simulated  annealing  was  very  time  consuming,  and  its  performance  on  this  highly  simplified  problem 
should  not  be  construed  as  ablanket  recommendation  when  scaling  up  to  the  real  world. 

Relationship  between  statistical  mechanics  and  combinatorial  optimization 

Statistical  Mechanics  Combinatorial  Optimization 

States  (of  system) 

Energy  (of  states) 

Ground  State 


Solutions  (to  problem) 
Cost  (of  solution) 
Optimal  Solution 
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Suggested  Approaches 

SCTD  should  add  a  neural  network  algorithm  to  their  set  of  algorithms  which  already  contains 
genetic  and  simulated  annealing  algorithms.  In  order  to  give  the  user  or  decision  maker  an  idea  of 
which  of  these  algorithms  might  be  most  suitable  to  meet  her/his  needs  in  the  design  process 
experiments  should  be  performed  similar  to  those  done  by  Mansour  and  Fox  (1991).  Performance 
criteria  should  be  solution  quality,  execution  time,  robustness,  bias,  and  parallelizability.  Perfor¬ 
mances  should  be  critically  evaluated  and  compared  for  a  number  of  examples  with  different 
topologies,  sizes,  and  granularities.  Following  the  Mansour  and  Fox  (1992)  lead  additional  algo¬ 
rithms,  the  parallel  versions  of  the  present  three  algorithms,  should  be  added  to  the  repertoire  and 
once  again  experiments  for  critical  evaluation  and  comparison  should  be  made.  Once  these 
experiments  and  their  results  are  known  it  will  be  easier  to  advise  the  user  as  to  which  algorithm  to 
use  for  a  particular  design  problem.  M.  Wann  is  recognized  as  a  neural  networks  expert  at  NAWC 
and  the  implementation  of  a  neural  nets  algorithm  should  present  no  difficulty. 

Additional  experiments  should  be  performed  with  a  new  model  to  the  design  problem 
where  the  individual  objective  functions  are  represented  as  a  vector  of  objective  functions. 
These  result  should  be  compared  to  the  weighted  sum  results  to  see  if  better  solutions 
can  be  obtained. 

If  the  Mansour  and  Fox  (1991  )  results  are  confirmed,  i.  e„  neural  networks  are  much  faster  than  the 
genetic  and  simulated  annealing  algorithms,  but  solutions  are  not  as  good,  then  one  should  consider 
a  situation  where  the  user  applies  the  neural  network  to  an  optimization  problem  and  then  takes  that 
solution  and  uses  it  as  input  (starting  point)  for  applying  the  simulated  annealing  algorithm.  This  is 
the  idea  of  the  Boltzmann  machine  or  the  Cauchy  machine  (Freeman  and  Skapura).  Speedups  and 
solution  quality  of  this  hybridization  might  be  of  interest  to  the  user.  Freeman  and  Skapura  (1991  ) 
suggest  the  possibility  of  using  classical  optimizing  techniques  once  near  a  minimum.  This  could 
be  presented  to  the  DM  as  an  option. 

Others  are  using  FORTRAN  for  implementation  of  these  algorithms  (neural  network,  genetic,  and 
simulated  annealing  algorithms  and  their  parallel  versions.)  Steir.berg  and  Digirolamo  do  not  even 
consider  using  Ada  as  the  language  of  implementation.  SCTD  is  restricted  to  Ada.  Would  the 
tasking  features  of  Ada  improve  the  implementation  process  for  these  design  problems? 

Although  Ada  is  designed  to  allow  30  tasks  concurrently,  actual  experiments  have  shown  a 
maximum  of  10  to  11  tasks  concurrently  in  reality.  FORTRAN  can  be  designed  to  run  many  more 
than  30  tasks  concurrently.  (1988  SIAM  Conference  on  Parallel  Computing,  Denver,  CO)  This 
deficiency  should  be  eliminated  in  Ada  9X. 

The  Wann  et  al  paper  does  not  specify  how  the  weights  for  the  composite  objective  function  should 
be  selected,  or  if  there  is  any  relationship  between  the  weights.  The  best  approach  might  be  to  have 
the  weights  sum  to  one.  This  would  follow  the  approach  of  Pareto  optimality  where  one  objective 
cannot  be  improved  without  worsening  at  least  one  other  objective. 

The  final  product  should  contain  pseudocode  and  data  flow,  or  data  structures  diagrams.  This 
would  help  with  maintaining  the  code. 
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The  simulated  annealing  and  genetic  algorithms  do  not  appear  to  guarantee  the  avoidance  of  local 
minima  A  suggestion  here  would  be  to  enhance  the  code  such  that  avoidance  of  local  minima  is 
guaranteed.  The  present  algorithms  do  seem  to  be  of  a  scalable  nature  This  would  need  to  be  taken 
care  of  if  the  project  is  to  be  expanded  arbitrary  topologies 
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